Recreation of spontaneous non-verbal behavior on a synthetic agent EVA

This paper presents a novel process of transferring the human-generated communicative behavior onto an embodied conversational agent. The aim of our work is to build a high-resolution motion dictionary based on empirical analysis of non-verbal behavior performed in multi-speaker informal dialogues. The verbal and non-verbal behavior is recreated by using this motion dictionary and on pure, unprocessed text. It involves animating hand and arm gestures, head movement and gaze, and facial expressions with lip-sync. The generated non-verbal behavior is speech driven and in synchrony with verbal information passed in form of synthetically generated speech. Key-Words: multimodal annotation, non-verbal behavior, human-speaker imitation, motion dictionary